Image search engine system with multi-mode results

ABSTRACT

An image search engine server having an image search engine that performs image searches based on a search term that is augmented by a built-in thesaurus and/or a dictionary. For a thesaurus-based algorithm, the approach is to send a query back to the user, who can select the image search domain, sub-domain, and other hierarchical search refinements from one or more dropdown menus. The items in the dropdown menus that the user selects during the “query back” are used to augment the search string entered by the user to better refine the image search. If the user entered search string is a single string of dictionary word or words or the dictionary mode is elected, then synonyms for that search string are used to generate the augmented search string for the final context-based search operation. The result is improved image search results.

CROSS REFERENCES TO PRIORITY APPLICATIONS

The present application is a continuation of U.S. Utility applicationSer. No. 13/487,967, filed Jun. 4, 2012, pending, which is acontinuation of U.S. Utility application Ser. No. 12/415,651, filed Mar.31, 2009, now issued as U.S. Pat. No. 8,200,649, which:

(1) is a continuation in part of U.S. Utility application Ser. No.12/185,796, filed Aug. 4, 2008, now issued as U.S. Pat. No. 8,190,623;

(2) is a continuation in part of U.S. Utility application Ser. No.12/185,804, filed Aug. 4, 2008, now issued as U.S. Pat. No. 8,180,788;and

(3) claims priority under 35 U.S.C. 119(e) to U.S. ProvisionalApplication Ser. No. 61/052,744, filed May 13, 2008, all of which areincorporated herein by reference in their entirety for all purposes.

BACKGROUND

1. Technical Field

The present invention relates generally to Internet searching, and moreparticularly to performing image searches over the Internet.

2. Related Art

Currently, text based search engines are normally used to find imagecontent over the Internet. When using text words or descriptorscorrelated to image file names or Meta data to find images, it isdifficult for a person who is not well versed in the search operation toget more focused and relevant search results. Too few words in the textsearch string lead to too many search results that are not of interestor relevant to the user. If the user tries to add more words into thesearch string, aiming to retrieve results of high relevance, there is atendency that the search results getting more unfocussed. It is oftenthe user's sole responsibility to construct an efficient search stringusing logical AND, OR, etc., operators which can help in retrieving morerelevant image search results, and the crafting of a sufficient searchstring may take many tries or may be frustrated entirely in the end. Fora novice user, without proper knowledge of using the search engine withsuch logical operators, it is simply impossible to perform efficientimage searching on the Internet.

Presently, search engines do not differentiate between a text and imagesearch. When the search results are presented to the user, contextuallyrelevant images maybe presented deep inside a huge or long search resultlist of hundreds or thousands of images. Under such situations, the usermay fail to identify the relevant images from the large search resultlist. This results in the user putting in a substantial amount of effortthat will then become more frustrating for the user during the imagesearch operation. Also, the isolated images that are also part of thetext pages may further complicate the search operation adding confusionto the user.

The current image search engines do not help in screening the imagespresented to a user in response to a text search in a manner that ismore contextual based on the use and environment where the pictureresides. When the user enters a search string for searching images ofhis concern or perspective, it may so happen that lots of irrelevantimages are also presented to him, based on the matching of a singleword, not the more revealing surround content. Most of the times, thepresented results will be so large that the user will fail to find thebest image, or any relevant image at all. This aspect of lack of focusand context awareness of current image search engines is very serious insome situations. For example, when young children are looking for someimages or pictures of their choice; it may so happen that they get adultimage content or porn pictures accidentally mixed in with relevantpictures, a very serious drawback to be dealt with in the current imagesearch algorithms.

Also, current search engines do not learn or understand what the userfind relevant or what a user is looking for by the way of searchinteractions. As a consequence of this the search engine cannot trackwhat the user is looking for during browsing a webpage. Thus, any eventthat happens during a search session such as user selecting a word, orphrase on the current webpage will not be considered by search engine todetermine what images or content the user may be interesting in. Due tolack of this feature, further refinement of the search string during asearch session is not possible. As a result of this the search resultswill often not be relevant to what the user is looking for.

There are some search engines which can query users only on arudimentary basis for selecting the specific search domains or areas,but the search operations performed are limited to those domains onlyand the search engines do not address the problem of optimizing relevantsearch results to the user. Such search engines cannot be considered asgeneral purpose search engines, whose objective is to generate morefocused search results from the web servers hosting images on the entireInternet, rather than a single or few limited domains of data/images.Basically, such domain-specific search engines are personalized searchengines, which do search operations only within the domains of someone'slocal interest but not to a broad user's interest. Thus, searchoperations leading to limited domains will not suit the purpose of theusers all the time, as what the user desires in terms of data and imagesmaybe elsewhere in a different domain or search space.

Current search engines do not maintain a centralized database and updateit periodically based on the routine search operations performed by theusers to make the search results more focused or making them morecontextual. Normally, visited sites through search operations aremaintained in a search database, but such format of storage in a searchdatabase cannot be utilized for making the search operation moreefficient during the subsequent search operations. Also, the merelink-based databases, which a search server maintains for normalInternet operation, cannot help in refining image search queries for theuser, thus a refined or augmented search string cannot be derived forfocused and efficient context-based image search operation. Therefore, aneed exists for a more effective and efficient way of searchingInternet-based image content.

Further limitations and disadvantages of conventional and traditionalapproaches will become apparent to one of ordinary skill in the artthrough comparison of such systems with the present invention.

BRIEF SUMMARY OF THE INVENTION

The present invention is directed to apparatus and methods of operationthat are further described in the following Brief Description of theDrawings, the Detailed Description of the Invention, and the claims.Other features and advantages of the present invention will becomeapparent from the following detailed description of the invention madewith reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a system block diagram illustrating a search engine thatsupports a context-based image search operation;

FIG. 2 is a block diagram illustrating the implementation of acontext-based search engine using a context manager, in accordance withone embodiment;

FIG. 3 is a block diagram illustrating the details of the functionalfeatures of a context manager of FIG. 2, in accordance with oneembodiment;

FIG. 4 is a perspective block diagram illustrating, the interaction of acontext manger with a client device of previous figures when the systemgenerates search context in accordance with one embodiment;

FIG. 5 shows a block diagram of an exemplary configuration of athesaurus for refining the user's query through a query-back window, inaccordance with one embodiment;

FIG. 6 shows a block diagram illustrating the outline of an image searchengine dictionary, in accordance with one embodiment;

FIG. 7 is a perspective diagram illustrating an augmented image searchstring generated by the context manager based on the image search enginethesaurus and dictionary lookup described per FIGS. 5-6;

FIG. 8 is a flowchart illustrating the method of operation of thecontext search engine for generating the search context using athesaurus (e.g., FIG. 5), or a dictionary (e.g., FIG. 6), in accordancewith one embodiment; and

FIG. 9 is a flowchart illustrating a method of incorporating one or moreuser responses into the algorithms driving the image search engine, inaccordance with one embodiment.

DETAILED DESCRIPTION OF THE DRAWINGS

Unlike any other search operation performed over the Internet, such astext searches, document searches, binary searches, etc., there is adearth of capable search engines for image content on the Internet.Those applications and programs that are available for searching imagesare not very effective as they simply match a text search string typedin by a user to a file name of an image or maybe some short metadataattached to the image file. If a user searches for “balloons,” expectingto find pictures of birthday party balloons, the search may return pageafter page of hot air balloon photos or weather balloon pictures simplybecause “balloon” is found in the name or short Meta data attached tothese pictures. If the user then limits the search to “pink balloons forkids birthday party”, the simple image search algorithms may not find alot of this text in the image file names or metadata, and return equallypoor information to the user. In effect, this form of searching ensuresthat the best pictures, cartoons, images, graphics, clipart, etc., ofkids balloons are missed by the user. Many users find that a picture oran image can convey information more effectively or be more pertinent tosearch than Internet text-based information. A photograph of an objectliving or nonliving may be a more useful piece of information to a userthan any text or audio file describing the object orally or through awritten language form. Also, other advantages of image transmissionacross the Internet are their ability to be significantlydata-compressed, as opposed to text messages. The application of imagesearch and processing finds application in several fields particularlyto give an example is, in security, crime investigations and lawenforcement, medical applications, on-line publishing, etc.

Further, education through the Internet is a very effective medium thatoften must involve learning through images, multi-media, videos,pictures, or other graphic mediums. Image based education can be moreadvantageous than education through a specific written or spokenlanguage, which fundamentally involves lots of text based processing andpresentations. The amount of information conveyed through text or audioprocessing can be much less effective and in the end more expensive thanlearning via multimedia, videos, images, pictures, or other graphicalcontent. When using graphical Internet-based education, one has to bevery careful with and concerned about the image content that isdownloaded, transferred, and accessed through Internet search operation.

Furthermore, most learning disciplines like Engineering, Science,Business, Management, etc., portraying observations, data, and conceptsin the form of graphs, charts, drawings, schematics, graphics, andimages each one of them likely are searchable on the Internet as images.It is the image or the pictures in all walks of human life thatsometimes allow us to most effectively communicate and learn. So,effective image searching, image processing, and knowledge disseminationthrough image content on the Internet is a valuable process.

The advancements of image processing technology have made machines ableto understand and enhance images, sometimes more capably and effectivelythan written human languages. This has enhanced the usefulness of imagesin all aspects of Internet communication, learning, and operation. Alsoother technologies hitherto infeasible have been made feasible due toimage processing advancement; for e.g. image sensor technology hasachieved low cost, high resolution integrated circuit chip cameras,which can capture movie frames at the required frame rate for highquality media, picture, and movie processing. It is the need of the hourto implement an Internet image search engine which does searching of theimages based on the context of the search to better capitalize on imagecontent over the Internet.

The space industry, military establishments, scientific community, etc.,requires real time image processing of image data shot from one or moresatellites. The cameras mounted on satellites shoot pictures at highresolution in real time and send them as frames, and the information isfinally stored as an images or image frames to generate movies,pictures, or multi-media. In all these situations, it is some form ofInternet (or intranet of a corporate company) infrastructure that ischiefly used to process, send, and receive images between groups ofpersons, etc. This processing requires maintaining huge image databasesand this data could be enhanced and rendered more useful by involvingimage search operations, necessitating the need of image search enginesthat are optimized for context-based image search.

There are standard image formats that have evolved over time and thatare available and make it possible to have unique standards to interpretimages or pictures over the Internet and through Internet searching.This makes it easy to derive or process information contained inpictorial or multi-media data for computer processing. Various pictorialformats may be easily understood by computers, thereby allowingcomputers to process, learn, and remember images efficiently, which issometimes required in certain applications.

Geographical distance between an image source center and an imagedestination center is not a large constraint given the Internet.However, most of the time searching images on the Internet is acumbersome and a complex task. Some of the search engines do searchingof both images and text/documents, like Yahoo or Google. In thatcontext, the search engines may be required to segregate image contentsfrom that of the text and document information. This necessitates andallows for a new generation of search engines that can search imagecontents efficiently and exclusively.

Currently, we need search engines that do image searching rather thanless concise and less useful searching of text and document-based webpages. A new search algorithm that enables image searching, based on thecontext, would be a useful tool on the Internet. Often, the screening ofimages needs to be done from text-based web pages. Also images fromcertain domains need to be screened from images in some other domains,for example, images from children sites need to be segregated from theimages on adult sites. There is also a need to maintain image databasesexclusively for the simplicity of maintenance and the ease of access.Images can be stored based on their relevance to a domain, category,etc., which helps ensure a quick and efficient retrieval of the imagedata during an image search on the Internet.

The ever-increasing need of picture and image exchange through theInternet has found its limitations from within the algorithms that makeimage search. As the Internet is the source of all sorts of information,whether it is text or image, it is the efficient and context-basedretrieval of this data that is highly essential. The context-basedsearch of images makes for the efficient utilization of the Internetinfrastructure, and this can reduce junk traffic on the Internet thatcontains redundant or irrelevant information during Internet searchoperation, as happens today. Search engines that perform image searchingvia a search context can provide efficient image search services togroups of people from different disciplines, irrespective of their levelof Internet search knowledge.

The embodiments taught herein facilitate various image search operationbased on the context of the search, thus resulting in more effective andefficient image searching. The image search engine can understand whatthe user requirement is, and the image search engine can accordinglymodify the user entered search string or the image search algorithm ordata access, to perform image search operation that better access onlythose images that better correlating with the modified or augmentedsearch string and therefore better correlate with the desires of theuser.

The context-based image search may require some internalreference/understanding that helps search engine in understanding thecontext-based searching. In one embodiment, a built-in thesaurus to thebrowser or search system is specially and optimally formatted. Thisthesaurus allows the search engine to learn, expand, and contract thecontext of the search and generate new or augmented search string(s) forperforming the image search operation in improved or expanded ways toassist the user. In order for the search engine to work as ageneral-purpose search engine over time for all users, the thesauruscontents may have to be extensive or exhaustive, and may need to beperiodically updated to remain current.

During an image search operation, a user enters a search string in theimage search engine. The search engine interacts with the user through aquery-back mechanism to refine the user's query (i.e. search string).The search engine queries back the user in a window providing a dropdownmenu containing a large list of all the domains from which the image canbe searched. The list of all the domains is maintained in the thesaurusin one embodiment. When the user selects one or more domains, the searchengine further queries user to identify the sub-domain to which thesearch string (or the image name) needs to be searched, by way ofpresenting him with a list of sub-domains derived from the thesaurus.Then user makes a choice of the sub-domain(s). Within one or morespecific sub-domain are lists of the categories, to which the searchstring needs to be searched from. Subsequent queries ask the user toselect a subcategory from within the selected category. Within asubcategory there may be many semantically-related searched objects withthe search string provided by the user. At each step the search stringprovided by the user will get augmented by domain, sub-domain, category,subcategory, names, etc. Finally, the augmented search list will befurther augmented with all the semantically-related search image objectnames that are present in a subcategory. Note, the hierarchy of domain,sub-domain, category, subcategory, names taught herein can be shorted tocreate a hierarchy of just domain and category, or domain-category-name,or a more complex hierarchy with more than 5 levels as discussed above.Therefore the number and detail of each query and the number and detailof the hierarchical construct that derived the query can change tocreate new embodiments within the scope of the embodiments taughtherein.

At end of all the query steps, the search string has grown far biggerand far more useful to the user for searching the image databases thanwhat the user had initially entered. This augmented search string isfurther constructed after each user response to the query or isconstructed at the completion of all queries, and the new augmentedsearch string is managed by a search engine module called a contextmanager, in accordance with one embodiment. Thesaurus lookup isperformed when the user-entered search string is a multiple word or aphrase, indicating the name of the image under search. If the searchstring is a single word from a dictionary or a single word parsed from amulti-word user entry, then a built-in dictionary lookup may suffice topick synonyms and generate an augmented search string by the contextmanager in accordance with the embodiment. Under a single word searchstring and in one embodiment, it is the user who will guide the imagesearch engine whether to do a thesaurus-based contextual image search ora dictionary-based contextual image search.

In one embodiment, user can also opt for direct image searching whereinthe image search will be performed directly based on the user-enteredsearch string (without being automatically further augmented by otherwords per a dictionary and/or thesaurus). In this situation, the searchresults are not context-based to begin with. This feature of limited thelevel of context-based machine-human interaction may be essential if theuser is not satisfied with the machine-assisted thesaurus or dictionarybased contextual image search operations or this feature may be usefulfor some other of the user's personal requirements. This direct imagesearch will be useful in the event that the thesaurus or the dictionaryhas not been updated periodically or maybe is available in the wronglanguage that the user intends to search with. A direct image searchoperation will also help in allowing previously unlisted images in thethesaurus to be listed into it. This can happen once, and henceforththat the image be accessible for everyone during further thesaurus basedcontext search.

FIG. 1 is a system block diagram illustrating a search engine orbrowser-based system that supports context-based image searchoperations. The perspective block diagram 101 of FIG. 1 is illustratinga search engine supporting the context-based image search operationsdiscussed previously. A plurality of client devices 103 iscommunicatively coupled to one or more image search engine servers 127,via the Internet 105. The Internet can be replaced with any manner ofconnection from one device to another device, such as opticalinterconnect, wireless interconnect, intranet, wireline, combinationsthereof, or other networks. The main components of the image searchengine server 127 are the network interface circuitry 105 that connectsthe search engine server 127 to external devices and the Internet, imagesearch engine 121, storage/memory 115, Operating system (OS) 107, Memory123, and the processing circuitry 125.

The image search engine 121 comprises a copy of the image search resultlist 117, often stored in its cache, main, random access, disk, ormagnetic memory, and the context manager 119 which produces imagesearches in a manner that is context-based on the user's text inputand/or responses to one or more server queries. If a thesaurus-basedcontext generation is opted by the user or server, or selected bydefault, the user will be queried back through query back window whichfacilitates a dropdown menu of domains, categories, etc., which aredefined in or in conjunction with the thesaurus. A Thesaurus-based imagesearch via thesaurus 111 will be the default option if the user entersan image search string that is not a single dictionary-based word or ifit is a multiple word phrase that requires more grammatical or complexprocessing to decipher. Otherwise, for simple or single word searchstrings, the image search can be conducted using the dictionary 113, bydoing a dictionary-based context search as opposed to a thesaurus-basedapproach. In some embodiments, the server may enable a dualthesaurus/dictionary approach, where the thesaurus parses more complexterms and sentences while the dictionary is consulted for smallerphrases and words to get more insight into the user search string.Therefore, in some embodiments, the thesaurus and dictionary can beaccessed simultaneously, together, or in series to finish search stringprocessing. In a thesaurus-based image search, semantically similarimage search object names from within a category or a subcategory of thethesaurus are picked by the context manager to add to or augment theuser-entered image search string. The image search engine 121 then usesthis augmented search string to perform the image search operation tofind more meaningful and relevant image material for the user.

For a single word or a default dictionary based words, adictionary-based context search using the dictionary 111 of FIG. 1 willbe selected and performed by the context manager 119 of server 127. Inthis situation, the dictionary meaning of the user entered single wordimage search string will be augmented by various synonyms or synonymouswords/phrases picked by the context manager 119 to aid in the search.Subsequently, the image search engine 121 does the image searchoperation and display the search result list to the user. If the userwishes to perform thesaurus based context image search to a single wordimage search string, then it is possible to do so by an option to selectthe thesaurus, instead of the default (for dictionary words) dictionarybased context.

The image search engine database 109 contains the table of the linksalready visited by several users over the Internet, in various earlierimage search sessions. If some other users request the same images whoselink(s) are already in the database will be readily displayed withoutneed to contact the respective web server which hosts that particularimage. These cached image links will be maintained for some specificduration after which they are deleted from the database for the storagespace requirement constraints. Memory unit 123 is the actual systemmemory used during any computation by the processing circuitry 125. Thememory is typically random access memory, dynamic random access memory(DRAM), static random access memory (SRAM), cache, magnetic diskstorage, non-volatile memory, electrically programmable read only memory(EPROM), electrically erasable read only memory (EEPROM), magneticmemory, optical memory, laser disks, other types of storage orcombinations thereof. The processing circuitry 125 is often one or morecentral processing units, microcontrollers, multi-core CPUs, graphicsprocessing units (GPUs), digital signal processors (DSPs), applicationspecific integrated circuits (ASICs), programmable gate arrays (PGAs),control logic, state machines, combinations thereof, or other processingcircuitry.

FIG. 2 is a block diagram illustrating the implementation of acontext-based search engine using a context manager, in accordance withone embodiment illustrated in FIG. 1. The context manager 223 (similarto context manager 119 of FIG. 1) is useful module for a context-basedimage search engine. To start image searching, users select the basiccontext generation mode in the image search engine instance window onthe client device. The basic context generation mode selections that auser has are thesaurus-based or dictionary-based based contextgeneration options and the direct search option. In other cases, a combothesaurus/dictionary mode is also available.

In most of the search operations, it is the thesaurus-based contextsearch that is used as a default, via the thesaurus augmented imagesearch string generator 221 within image search engine 227. The userenters the image search string in the image search engine window. Inresponse to this entry, the context manager 223 will be executed on theimage search engine server to grab all the domain name information fromthe thesaurus, collect that information into a dropdown menu or similardisplay mechanism, and present the information to the user. In somecases, the thesaurus processing will require more complex processingthat simple a table look-up for similar or synonymous words, as may bethe simpler case for the dictionary mode of operation. The thesaurusprocessing can use grammatical processing similar to that used in modernword processing programs to parse grammar, structure, and meaning froman entered string of information. In some cases, the thesaurus may callupon the dictionary to aid in its processing. In other cases, thethesaurus is simply a hierarchical organization of search domaincategories and sub-categories that a user can select within to refine asearch to a more focused and relevant image/data domain. Afterprocessing and displaying the domain name information, the user selectsone or more domain names from the dropdown menu, based on his knowledgeof the image searching he requires or with some judicious startingguesswork. Subsequently, the image search engine learns thisinformation, and instantly the context manager modifies the image searchstring by one level to incorporate searching of the selected domain setsof data. Thereafter, the context manager interacts with the thesaurusand grabs all the sub-domain names under the previously selecteddomain(s) and provides another dropdown menu to the user. User repeatsthe selections of sub-domains as previously discussed for domains. Thecontext manager modifies the user image search string by another levelby adding the sub-domain name/information to the search query providedby the user. This process is repeated until the image search string ismodified by all the levels based on the number of levels of imagedatabase granularity that are built into the thesaurus. In the examplecase discussed herein, four levels of hierarchical structure areconsidered by the thesaurus, and these levels are the domain level,sub-domain level, category level, and subcategory level for thesimplicity of illustration. In each level, less image data, but moreinterrelated or relevant image text data is pooled together andassociated with one another for more effective searching of denselypopulated relevant content.

In a large database, the number of levels in the thesaurus can furtherbe nested shallower or deeper that described above to accommodatevariety of the image object types that are coming from variety ofdifferent fields. In the last level of the hierarchical tree datastructure, all words or phrases that a similar or synonymous with thesearch requested by the user are resident. The context manager picks allof the names corresponding to the respective images and adds them topreviously modified image search string. The names or words in the lastlevel are all semantically related words, and might not be simplysynonymous during the thesaurus based context generation.

If the option of dictionary-based searching is chosen earlier, then thedictionary-based context will be generated, via the dictionary augmentedimage search string generator 225 within image search engine 227 of FIG.2. This option works for the search strings that can be parsed forsimple dictionary meaning in a relatively simple manner. The dictionaryshould be of special format where each word in the dictionary has addedto it all the synonyms or related words or phrases for each word. Thishelps the context manager to pick those synonyms automatically and addto the user entered search string, thereby generating the augmentedimage search string that can be used by the server 229 to perform theimage search. The image search engine uses the augmented image searchstring for searching all the relevant images from the Internet, whichwill have high probability of context relevance to what the user islooking for.

It is also possible to allow the dictionary and thesaurus queryingmechanism to elect phrases or words that should not be search or notincluded with the search results. For example, if someone is searchingon the word “Ram,” this word could mean an animal, a rock bank, aportion of an oil rig drill bit, or random access memory, for example.By recording these common inaccuracies in the thesaurus data structure,the server can help the user, through queries, filter out content areasthat they will certainly not be interested in viewing or using. Theblock diagram 201, of FIG. 2 is illustrating the implementation of acontext-based search engine using a context manager 223, in accordancewith one embodiment. The client device 211, has an instance of the imagesearch engine 205 opened in a network browser window 203. The mainfeature of the context-based image search operation is the query-backwindow 209 that is presented to the user one or multiple times inresponse to the user's entry of the image search string 207. The clientdevice is communicatively coupled to the image search engine server 229(server 127 of FIG. 1 repeated), via the Internet 213 (same as theInternet 105 of FIG. 1 repeated).

The context manager 223 (same as context manager 119 of FIG. 1 repeated)is shown in more detail in FIG. 2. The context manager 223 of FIG. 2contains the context generation features/modules that are selected andused based on the user's choice or machine/control panel defaultsettings opted at the beginning of the image search session or set up byIT specialists or the user. This selection usually enables one mode,such as thesaurus-based or a dictionary-based search context generation.The thesaurus augmented search string generator 221 is a module withinthe context manager 223 that is executed when the user enters an imagesearch string and thesaurus-based context searching is enabled. If adictionary context option is selected, then the dictionary augmentedsearch string generator 225 module within the context manager 223 isexecuted when the user enter enters an image search string. Once theaugmented image search string is generated by one or more of generators221 and 225 through interactive queries with the user, the image searchengine 229 will perform the search operation on the Internet to searchand deliver the contextually relevant images. The thesaurus 215(thesaurus 111 of FIG. 1 repeated) and dictionary 217 (same asdictionary 113 of FIG. 1 repeated) in the storage 219 (same as storage115 of FIG. 1 repeated) are constantly updated to reflect the expansionor addition of new image information or links under newer domains or newhierarchies in the thesaurus' text structure.

FIG. 3 is a block diagram illustrating, the details of the functionalfeatures of a context manager, in accordance with one embodiment. When auser enters an image search string, the image search engine on the imagesearch engine server should receive the search string to enableprocessing of it. In response to the user entering the image searchstring, the image search engine will present a query-back window to theuser that essentially presents a dropdown menu displayed to the user.Subsequently, the user responds by one or more selections that are madein the query back window. The context manager will receive theresponse/selections from the client device on which the user isperforming the image searching. Further, the context manager takes boththe user provided image search string and the query back window responseto generate the augmented image search string. The augmented searchstring is then used to provide more comprehensive, meaningful, relevantsearch operations that provide more comprehensive, meaningful, relevantimage search results to the user for his search string.

The block diagram 301 of FIG. 3, is illustrating in more detail thefunctional features of a context manager that enables the search engineto perform context-based searches using the built-in thesaurus as thecontrol module, in accordance with one embodiment. The image searchengine server 305 (server 127 of FIG. 1 repeated) is running imagesearch engine 303 (similar to image search engine 121 of FIG. 1repeated), and retrieving all the necessary information from a pluralityof the client devices as shown in FIG. 1. The context manager 307(manager 119 of FIG. 1 repeated) generates the necessary context andaugmented search terms for a more focused retrieval of the requestedimage search results.

The context manager has an image search string receiver 311 that isinterfaced to by a user operating on the client device. The userinterfaces with receiver 311 via the network browser window in which animage search engine instance 205 of FIG. 2 is running. Further, the userresponds to the query back window by selecting one or more image searchdomains of his choice that related or correlate to his search string,and communicates these selections to the image search engine 303, on theimage search engine server 305, via a user response receiver module 313in FIG. 3. The augmented image search string generator 309 assembles theuser-provided image search string (captured by receiver 311) with userresponse data (captured by receiver 313) resulting in the augmentedimage search string. The augmented image search string is then used bythe image search engine to perform the relevant image search operation.Finally, the image search result lists, thumbnails, pictures, or otherdata are presented back to the user on the client machine 211 of FIG. 2.In this process, the image search engine will retain a copy of thesearch results 117 of FIG. 1 to update its image search database.

The thesaurus-context generation involves “query back” operations andsubsequent response(s) from the user, in accordance with one embodiment.When a user makes a decision whether the search is to be conducted usingthe thesaurus-based context or a dictionary-based context, by opting atthe beginning of the image search session from the image search enginewindow on the client device for example, the context manger will becomeaware of what routines to execute and/or what data to load into thememory (from storage), whether thesaurus-based and/or dictionary-based;The thesaurus programs and data are loaded and executed forthesaurus-based context generation, and the dictionary programs and dataare loaded for using a dictionary for dictionary-based contextgeneration.

First, the user inputs the image search string using the client deviceor another input device. The image search string receiver 311 of FIG. 3receives the image search string entered by the user through networkinterface circuitry and the Internet or a similar network. Subsequently,the context manager queries back user via a window having a dropdownmenu or a like mechanism for receiving user input and provides theentire listed top level search domains or at least the search domainsthe server determines to be useful to the user given an analysis of thesearch string. The user responds by selecting one or more of the toplevel search domains to confine his search domain to specific searchcriteria, data, and text that is of most relevance to the user's currentneeds/desires. The user's response receiver 313 of FIG. 3 receives theuser's response for the “query back” queries, and generates an augmentedsearch string or augmented search data after each step of the user'sresponse or at the end of the user's multi-tier hierarchical input. Afinal augmented image search string will be generated and available foruse at the end of all “query back” operations, and the server will usethis final augmented image search string to perform the context searchrequested by the user.

FIG. 4 is a perspective block diagram illustrating, the interaction ofthe context manger resident on the server with the client devicesillustrated in FIG. 1 when generating the search context data inaccordance with one embodiment. The context manager 407 (manager 119 ofFIG. 1 repeated) understands the context and requirements of the search,based on the user's interactions and input through the original searchstring 405 and responses to various queries 403. The image search string405 (similar to search string 207 of FIG. 2 repeated) comes from theclient device 211 of FIG. 2 in response to which the context manager 407generates a series of very simple queries via a query back window 403,wherein the user simply should to select one or more categories or namesfrom the dropdown list to satisfy the queries for the server software.The user's response through query back window 403 is submitted back tothe context manager 407 as shown in FIG. 4. In response to the userinput, the context manager 407 can generate the augmented image searchstring 417 (in a thesaurus-based manner). For the thesaurus-basedprocess, the server uses the image search string 405 and thesemantically nearer terms 411 picked from the thesaurus 409 andavailable in the storage 419.

If the user had opted for the dictionary-based context search at thebeginning of the search session or if the user entered an image searchstring which is a dictionary word, the context manager 407 may derivemany synonymous words simply from accessing the dictionary 413, existingin the storage 419. The context manager 407 can assemble or associateall or several of the synonymous words picked from the dictionary withthe search string entered by the user, resulting in an augmented imagesearch string 421 or search data structure/database 421 (dictionarybased). The image search engine 121 of FIG. 1 does the subsequent searchoperations for relevant images based on the augmented search string 421.Also, the server may be able to use a hybrid process of a combination ofthe thesaurus and the dictionary where the processing of the dictionarycan aid in the thesaurus processing and vice versa, to generate a hybridaugmented image search string.

FIG. 5 shows a block diagram of an exemplary configuration of athesaurus for refining the user's query through a query-back window, inaccordance with one embodiment. For an effective image search, thestructure or configuration of the thesaurus is used for categorizing andlisting a huge number of names of the various search objects on theInternet, based on their semantic relations. Basically, the thesaurusorganizes graphical, picture, video, artistic, schematic, and otheraudio-visual information according to relevance to each other in a treedata structure. As a simple example, a domain may be “dogs”, with twosubdomains being “small dogs” and big dogs”, with the sub-domain “smalldogs” having two categories “dachshund” and “toy poodle”, with “toypoodle” having categories such as “black toy poodle”, “white toypoodle”, etc., and wherein the search object links, files, or pointersare grouped at the leaf nodes of the tree data structure under searchobjects. Note that other data structures like linked lists, B trees,etc., can be used to structure, and store information for the thesauruselements taught herein. In essence, the thesaurus is something similarto a telephone directory where all the telephone numbers aresystematically grouped according to various relationships and listedaccording to their relationships, so that for even a layman can pick arequired telephone number by traversing the data structure viauser-answered queries.

The image search engine thesaurus will have a list of top-level domainnames, and below the top-level domain names there are severalintermediate levels with each level item having its own next level listof sub-domains. The lowest level has all the sub-domains expanded intosemantically related search objects, via files, pointers, allocatedstorage, identifiers, and/or other constructs. One can imagine the imagesearch engine thesaurus as a huge tree data structure, with its trunkrepresenting the thesaurus itself and the emerging branches as thedomains, and each branch further branching into sub-domains (smallerbranch lets), and so on until finally the leaves of the tree are thesemantically related search objects/lists/nodes as the final structuresin the tree database.

Block diagram 501 of FIG. 5 is an exemplary configuration of a thesauruswherein a user selects the domain, category, etc., through userresponses to a query back window that has a dropdown menu to helprefine, focus, and narrow down the user's search, in accordance with oneembodiment. FIG. 5 is an example configuration of a thesaurus with just4 levels (and any number of levels from 1 to N is possible, where N isany integer). In FIG. 5, only 4 levels are indicated for the simplicityof explaining the concept of present invention. In addition, even if Ntiers or hierarchies are used, the system may not require a full Nnumber of queries to traverse the tree. many different data structuresexist to obtain an N level hierarchy, but settle into all the data thatthe system needs in less than N user queries (e.g., the user enteredinitial search string may already allow the thesaurus to confine thesub-tree structures and nodes of the thesaurus that are pertinent to theuser). At the first top level of the data structure in FIG. 5, we havethe Domain-1 505 to Domain-N 509 illustrates, and these domains indicatea plurality of different top level domain names. The next second levelof the hierarchy is indicated under the domain-1 505 in this example,and this next tier of the hierarchy is shown as sub-domains ranging fromSub-domain-1 511 to Sub-domain-M 515 where M is any positive integer.Similarly, all the remaining domains in the top level will lead to aplurality of sub-domains as indicated under the domain-1 505; however,these are not specifically shown in FIG. 5 for simplicity ofillustration. A Level further below the Sub-domains 511-515 is the thirdlevel in the hierarchy of the data structure containing the categoriesillustrated as category-1 517 to category-K 521, where K is any positiveinteger. Similarly, all the remaining sub-domains in FIG. 5 will lead toa plurality of categories as indicated under the sub-domain-1 511, butsuch is not shown for ease of illustration. At the fourth level, we havethe subcategories levels/nodes. Category-1 517 at the third level hassub-category-1 523 to sub-category-I 527 illustrated in FIG. 5, where Iis any positive integer. Similarly all the remaining categories at thethird level will lead to a plurality of subcategories as indicated underthe category-1 517, but such is not illustrated in FIG. 5 for simplicityof illustration. As the data focused in each sub-branch of the treestructure are more interrelated and the domain size of linked imagestherein gets progressively smaller, the search results are accordinglymore focused and relevant as you move down the tree structure. At somepoint, the user may want to stop the traversing through the tree wherethe user is at, and not refine the search further. This is possible, andif the user wants to stop at sub-domain 511 for example, or stop atcategory-2 519, then the thesaurus can gather all the relevant augmentedsearch term data from all the nodes underlying and connected/associatedwith the node on which the user stopped the query process. The lastlevel is shown to emerging from the subcategory-1 523 in FIG. 5, andthis tier of the hierarchy has the items labeled search-object-1 529 tosearch-object-J 533 where J is any positive integer value. The data atthis level contains a plurality of the search object names, identifiers,links, meta data, etc. which the image search engine can use to createan augmented search data string or data structure to do searchingoperation (in the augmented search string) that have more favorable andrelevant results for the user.

In one embodiment, the search objects items labeled search-object-1 529to search-object-J 533, will directly be the links of certain imagesearch sites, text descriptors of interest, metadata, images themselves,or some mixture of many different forms of data type. For example,sometimes the objects 529-533 are the list of links corresponding torestaurants, or shopping centers, etc. If a user for instance is lookingfor a list of restaurant images, the list of links can readily bedisplayed to him, instead of searching across the entire Internet forthe same search results links. Often, links, addresses, htmlidentifiers, or pointers will be used, as this results in more efficientprocessing and space utilization within the host of severs that helpconstitute the Internet. Note that the nodes in the tree structure canconnect in a cross-connected manner in some embodiments. For example, acategory domain of mammals and a domain of dogs may link through thehierarchy to similar leaf nodes or final data in the end. Therefore, insome data structures, the connections on one node or to certain dataitems in a node can be shared between many parent nodes in the treestructure of FIG. 5.

FIG. 6 shows a block diagram illustrating the concept of an image searchengine dictionary, in accordance with one embodiment. In addition to thethesaurus-based context search, a dictionary-based contextgeneration/search is available for the search operation. The dictionarybuilt into an image search engine has a basic structure or outline asdepicted in FIG. 6, in accordance with one embodiment. For each wordthat a user could input as a part of a search string, there are a numberof synonymous words listed in the image search engine dictionary 603.When a single word image search string is entered by a user or a simpleshort phrase that can be easily parsed into a finite number of simplewords, the context manager 119 of FIG. 1 will look for all the synonymsof the words on the search string that are listed in the dictionary 603.Based on the matching and inclusion of synonymous words, phrases, anddata, an augmented search string is generated. With the augmented imagesearch string now created, the image search operation can be performedand delivered to the user using the enhanced or augmented image searchstring from the dictionary.

In one embodiment, the dictionary-based augmented image search stringwill further be modified to include more context with the help ofthesaurus lookup as explained in FIG. 4 and FIG. 5, earlier. This mayneed to be done because an augmented search string formed by just usingthe dictionary lookup based image search strings/information may notyield adequate contextual image search results, as the image searchstring may not have sufficient contextual focus if it is too big or toosmall.

Block diagram 601 of FIG. 6, illustrates the basic structure or outlineof an image search engine dictionary, in accordance with one embodiment.The dictionary 603 contains words, likely listed in the alphabeticalorder or in some other formal structure, along with the list of theirsynonyms that is associated with each word. The example dictionary inFIG. 6 shows word1 605, word2 607, word3 609, etc., in alphabeticalorder or some other formal order, where these words are words that auser can enter as part of a search string. The context manager 119 ofFIG. 1 looks into the dictionary 603, finds one or more of theuser-entered search string words and uses one, some, or all its synonymsfrom the dictionary to generate the augmented image search string.

FIG. 7 is a perspective diagram illustrating the augmented image searchstring generated by the context manager and based on the image searchengine thesaurus and/or dictionary lookup operations. A simple buteffective approach to generate the image search context is to augmentthe user supplied image search string. The augmenting words are variouswords that will narrow down the search domain or space to a more finiteand relevant set of image search results. According to one embodiment,the words that narrow down the image search results based on the contextare the various levels of domain, sub-domain, etc., names in the searchengine thesaurus. In the dictionary based image search contextgeneration, these words may be one or more synonymous words derived fromthe image search engine dictionary, augmented with the user enteredsearch string, which narrow down or better focus the search resultsbased on the context of the search operation.

An example of augmented image search strings constructed based on FIG. 5(thesaurus) and FIG. 6 (dictionary) may be created by the contextmanager and may be structured as shown in the FIG. 7 (a) and (b)respectively. In FIG. 7 (a), the (final) augmented image search stringis constructed based on the user's response from the client device tovarious query back operations from the image search engine. Theuser-entered image search string 703 will be augmented by the domainname 705, which the user selects from the dropdown menu at the firstquery back, resulting in the “first augmented image search string.” Thesub-domain name 707 selected by the user from the dropdown menu duringthe second query back will be augmented to the “first augmented imagesearch string,” resulting in a “second augmented image search string”.The category name 709 selected by the user from the dropdown menu duringthe third query back will be augmented to the “second augmented imagesearch string”, resulting in the “third augmented image search string.”The subcategory name 711 selected by the user from the dropdown menuduring the fourth query back will be augmented to the third augmentedimage search string, resulting in the “fourth augmented image searchstring.” Finally, the list of search objects name 713 from within aselected subcategory from the thesaurus will be augmented to the “fourthaugmented image search string,” resulting in the “final augmented imagesearch string.” Or, in the alternative, some or all of the queries canbe captured by the client or the server and only processedintermittently or at the end of the query process to create a finalaugmented search string. A perspective picture of such a “finalaugmented image search string’ format is depicted in FIG. 7 (a), finallywhich is to perform the image search operation by the image searchengine.

In FIG. 7 (b), shows a final augmented image search string, constructedby dictionary lookup. User entered image search string 715 which isnormally a dictionary-based word (for the dictionary-based contextgeneration), will be augmented by the synonyms of the user enteredsearch string 717. The composite of the two strings viz. 715 and 717 isnow the “final augmented image search string” of FIG. 7 (b), which thesearch engine uses to perform the image search operation. In oneembodiment, this “final augmented image search string” can further beused as user entered image search string at 703 of FIG. 7 (a), forfurther refinement of the context.

It is important to note that some algorithms and embodiments can querythe user to identify just those few key synonyms or thesaurus-basedconstructs that the user wants out of the structure of FIG. 7. And, theserver can ask the user to augment the logic of the search using and,or, not or other logic operators. For example, the user may wantpictures of new BMW cars, not used ones, from the year 1987. In thiscase, the algorithm could identify from the thesaurus, dictionary orsearch that the search should be /“Bavarian motor works’, “BMW”, 1987,“new car”, “new auto”, “new automobile”, “new vehicle”/ and the user maybe able to modify the search through queries or server assistant to say/“BMW” and “new” and “1987” and (car or auto or automobile or vehicle)not “used”/. Meaning, it may not be enough to simply append severalrelated words together, but to process the context of the query to getonly the needed/relevant words assembled into the right logical contextto perform the search well.

FIG. 8 is a flowchart illustrating the method and operations 801performed by the context image search engine, wherein the search contextmay be generated using a thesaurus or a dictionary. If none of theoptions processing enabled by the thesaurus and dictionary are made, thesearch engine will perform a direct image search based on the searchstring entered by the user in a right-hand vertical path shown in FIG.8. In accordance with the present invention, three different imagesearch scenarios are considered for generating the context for the imagesearch on the Internet. It is sometimes important to maintain andprovide flexibility for the Internet users that do image searches on theInternet. Flexibility will allow for the best algorithm to beimplemented for the user's needs and allow the server to provide imagesthat are close to what they are looking for on the Internet, rather thanconfusing them with search operations that present them a disordered andchaotic image search results.

A search algorithm based within the image search engine may not performcontext-based searches for all types of the search strings used by theuser under all the search scenarios. If the user wants the search to bedirected based on whatever he enters as the search string, this optionis made available as one of the options available to the user. Thisoption is in addition to the context-based image search using thethesaurus-based algorithm (see left hand vertical column of FIG. 8) andthe dictionary-based algorithm (see the middle column of FIG. 8). In theflowchart of FIG. 8 the operation of the image search engine forthesaurus based context generation for image search is explained withthe four levels hierarchy based thesaurus at FIG. 5.

The flowchart illustrates the method/operations 801 performed by thecontext search engine, wherein the context may be generated using athesaurus or a dictionary approach. In one embodiment, after starting atthe image search engine at a step 803, a user can chose one option outof three: a thesaurus based search algorithm/image as in steps 805-821,dictionary based search algorithm/image search as in steps 823-831, or adirect search algorithm or image search as in steps 833-839 of FIG. 8.

If the thesaurus context search option is chosen at a step 805, theimage search engine receives the image search string at a step 807, viauser input and interfacing from the user via one or more client devices.The image search engine sends one or more queries back to the user atthe client device, prompting him to select the domains names asexplained in the FIG. 5 starting at step 809. As the servers receivessuccessive responses from the user for queries sent by the image searchengine, the server augments the image search string with the domain nameat 811, sub-domain name at 813, category name at 815, and subcategoryname at 817 as shown in FIG. 8 and further shown in a data structureformat previously in FIG. 7. Finally, the search engine augments thesearch string obtained at step 817, with semantically related wordslisted at the subcategory level of the thesaurus via a step 819,resulting in the final augmented search string used for context-basedimage search by the image search engine via a step 821.

If the user opts for the dictionary-based context search at 823, theuser enters an image search string this is a single dictionary word or asimple concatenation of a few dictionary words. The user-entered searchstring that is to be processed via the dictionary is received by theimage search engine at a step 825 from the user on a client device. Uponreceiving the image search string, the context manager 119 of FIG. 1derives the synonymous words for the words in the search string byperforming dictionary lookup(s) at a step 827. Subsequently, the searchengine context manager generates an augmented image search string at astep 829. The image search engine performs an image search operation ata step 831 and presents the image searches to the user. In oneembodiment, after the construction of the augmented dictionary basedsearch string at 829, the search engine will prompt the user to performfurther thesaurus based context generation through query back window.This process can create a more focused or relevant search string orconstruct, as explained in the previous paragraphs.

If the user has opted for direct search as in steps 833-839, the imagesearch engine does the image search operation with the image searchstring as entered by the user at a step 835. This option will behelpful, if the entire Internet needs to be searched for the image,which can be of particular interest for the user. The search enginereceives the user-entered string as at a step 837, and the image searchengine performs the search operation over the entire Internet via a step839 to display the search results to the user on the client device.

FIG. 9 is a flowchart illustrating the method performed by the user inresponse to the queries sent by the search engine server. This processallows the user to select a multiple choice of, domains, sub-domains,categories, subcategories etc., and names in accordance with oneembodiment and allow the user to assist the server in arranging thevarious selected data in terms of what is included, versus excluded,plus logical search context (or, xor, and, not, within X words from,etc.) in some embodiments. In the process of generating a context forthe image, the image search engine initiates a series of interactionswith the user. This interaction is triggered by the user's entry of theimage search string, with an option made in some manner to either gowith thesaurus and/or dictionary based context (if any). Subsequently,user will be guided through various interactive steps so that he willselect items from the dropdown menu and properly arrange one or more ofcategories, domains, context, logical connections, etc.

After the user or system makes a selection of type of context generation(thesaurus and/or dictionary) that will be used, the process willcontinue. In accordance with an example of a thesaurus configuration asdiscussed in FIG. 5, the user responds to the queries sent by imagesearch engine by simple mouse selections from the dropdown menu in thequery back window. For example, in four interactions that occur insequence, the user performs domain selection, sub-domain selection,category selection, and subcategory selection. If the user elects thedictionary context or the direct search options chosen, user has just toenter the image search string and do only minor (if any) dictionaryquerying for inclusion of terms, exclusion of terms, and logicalconnections between terms. The context manager takes care of subsequentsearch operation, as discussed in FIG. 6 and FIG. 7, earlier.

The flowchart 901 illustrates the method performed by the user inresponse to the queries sent by the search engine of FIG. 1. The userenters the image search string at a step 903. If the thesaurus-basedcontext generation option is chosen by the user or the system at thedecision box 915, the context manager 119 of FIG. 1 will send a queryback window to the user or client device, prompting the user with thetop-level domain names extracted from the thesaurus. Then, the user willmake one or more domain name selections via mouse or keyboard selectionat a step 905. Subsequently, the context manger presents one or morequeries that prompt the user to select a name of the sub-domain from adropdown menu for which user does mouse selection of one or moresub-domain names at a step 907. Next, context manger 119 of FIG. 1prompts the user with a dropdown list of category names, and the userresponds by mouse selection of one or more category name at a step 909.Further, the context manager present user with a dropdown list of allthe subcategories within the previously selected category. The user doesthe subcategory selection(s) at a step 911. The search engine contextmanager derives all the semantically related words within thatsubcategory and generates the augmented search string as discussed withrespect to FIG. 7, and subsequently does the context-based image searchoperation at a step 913.

At step 915, if the thesaurus-based context is not opted, the user willbe looking for the dictionary-based context generation, or this optionmaybe selected by default if the user entered an image search stringwhich is a dictionary based word or simple phrases of words. In thiscontext, the context manager of the image search engine of FIG. 1receives the user entered image search string at a step 903.Subsequently, the system derives the synonymous words from one or moredictionary elements at a step 919 in response to the user entered imagesearch string. An augmented image search string is then generated usingthe synonyms derived from the dictionary at a step 921, as explained ingreater detail in FIG. 7. Finally the image search engine performs theimage search operation at 923 and presents the image search results tothe user, likely displaying them on the client device for review by theuser.

At a step 917, if the dictionary option is not made in some form, it islikely the direct search option that has been chosen by the contextmanager of the image search engine. This option of the image search onlyrequires whatever the image search string a user enters from the searchengine instance 205 of FIG. 2 from the client device 211 of FIG. 2. Uponreceiving the user entered image search string, the image search engineperforms the image search operation at a step 925 using only the imagesearch string, and the results are presented to the user by displayingthem on the client device 211 of FIG. 2.

The thesaurus method taught herein can be used for other purposes. Forexample, the thesaurus hierarchy method may be used to process pictureson the Internet and their surrounding context. If, for example, there isa picture associated with a web page, text, or other content that can beprocessed using normal grammar processing, then the thesaurus algorithmcan process text and information around the picture to get a better ideaof what is in the picture. For example, assume there is a picture on awebsite or news article. We may not know what the picture is and imagerecognition processing on the pictures and metadata scanning may fail ornot be revealing. In this case, the server, which is processing picturesfor inclusion into the right node, hierarchy, category of the datastructure of FIG. 5 may process surrounding information and text thatisn't necessarily more picture data to derive more meaning from thepicture. For example, if the text around the picture says, “themotorcycle accident was severe as show in picture 1-1” or if the htmlindicate some text like “the motorcycle accident was severe” with closeproximate reference to a picture with a label in the html or xml, thenthe server can infer what is in the picture. If the server can analyzesurrounding data in this matter and accurately infer what is in thepicture, such material can be properly placed into the data structure ofFIG. 5 and used for processing of image searches, even though the imagemay be a “black box” or a data item from which we can glean no directinformation.

As one of ordinary skill in the art will appreciate, the terms “operablycoupled” and “communicatively coupled,” as may be used herein, includedirect coupling and indirect coupling via another component, element,circuit, or module where, for indirect coupling, the interveningcomponent, element, circuit, or module may or may not modify theinformation of a signal and may adjust its current level, voltage level,and/or power level. As one of ordinary skill in the art will alsoappreciate, inferred coupling (i.e., where one element is coupled toanother element by inference) includes direct and indirect couplingbetween two elements in the same manner as “operably coupled” and“communicatively coupled.”

The present invention has also been described above with the aid ofmethod steps illustrating the performance of specified functions andrelationships thereof. The boundaries and sequence of these functionalbuilding blocks and method steps have been arbitrarily defined hereinfor convenience of description, and some features may be merged, splitdifferently, or further segmented in other representations. Alternateboundaries and sequences can be defined so long as the specifiedfunctions and relationships are appropriately performed. Any suchalternate boundaries or sequences are thus within the scope and spiritof the claimed invention.

The present invention has been described above with the aid offunctional building blocks illustrating the performance of certainsignificant functions. The boundaries of these functional buildingblocks have been arbitrarily defined for convenience of description.Alternate boundaries could be defined as long as the certain significantfunctions are appropriately performed. Similarly, flow diagram blocksmay also have been arbitrarily defined herein to illustrate certainsignificant functionality. To the extent used, the flow diagram blockboundaries and sequence could have been defined otherwise and stillperform the certain significant functionality. For example, FIG. 9illustrates that a detection of thesaurus-based processing happensbefore dictionary based processing. It should be clear that one couldperform the same basic algorithm by changing this order and performingdictionary detection before thesaurus detection in some embodiments.Such alternate definitions of both functional building blocks and flowdiagram blocks and sequences are thus within the scope and spirit of theclaimed invention.

One of average skill in the art will also recognize that the functionalbuilding blocks, and other illustrative blocks, modules and componentsherein, can be implemented as illustrated or by discrete components,application specific integrated circuits, processors executingappropriate software and the like or any combination thereof. Also,thesaurus, as used herein, is intended to mean any module, software,system, or process that takes a search string or input from a user andprocesses it with feedback from the user to refine the search algorithmor data before or during searching using some form of data structure orcomputer data and/or query system to refine the search. The process anddata structure that enables this image searching can be changed andarrive at similar improved search results for the user. The functionalblocks and processes described herein can be performed by customsoftware, firmware, custom hardware, a general purpose CPU, otherexecution engine, or some combination thereof.

Moreover, although described in detail for purposes of clarity andunderstanding by way of the aforementioned embodiments, the presentinvention is not limited to such embodiments. It will be obvious to oneof average skill in the art that various changes and modifications maybe practiced within the spirit and scope of the invention, as limitedonly by the scope of the appended claims.

1. An image search system that communicatively couples with a pluralityof client devices that deliver a corresponding plurality of originalsearch strings, the image search system comprising: a processinginfrastructure that receives a first search string and a search modeidentifier from a first device of the plurality of client devices of afirst user, the search mode identifier being selected by the first uservia an offering presented by the first device, the offering includingeach of a non-augmented search, a synonym augmented search, and a searchaugmented with at least one group of differing meaning text; theprocessing infrastructure, based on the search mode identifier, performsa search of text, the text being associated with a plurality of imagesgathered via a web crawling process; and an interface through which theprocessing infrastructure delivers search results to the first of theplurality of client devices, the search results corresponding to thesearch mode identifier,
 2. The image search system of claim 1, whereinthe processing infrastructure further receives restriction input fromthe first user and the offering further including a search limited bythe restriction input.
 3. The image search system of claim 1, whereinthe search results are limited by at least one adult filter parameter.4. The image search system of claim 1, the search server furtheraugmenting the first search text by a context manager.
 5. The imagesearch system of claim 1, wherein the synonym augmented search is basedupon a dictionary approach.
 6. The image search system of claim 1,wherein the search augmented with at least one group of differingmeaning text search is based upon a thesaurus approach.
 7. An imagesearch system that communicatively couples with a plurality of clientdevices that deliver a corresponding plurality of original searchstrings, the image search system comprising: a processing infrastructurethat receives a first search string and a search mode identifier from afirst device of the plurality of client devices of a first user, thesearch mode identifier being selected by the first user via an offeringpresented by the first device, the offering including each of a synonymaugmented search and a differing meaning search; the processinginfrastructure, based on the search mode identifier, performs a searchof text, the text being associated with a plurality of images gatheredvia a web crawling process; and an interface through which theprocessing infrastructure delivers search results to the first of theplurality of client devices, the search results corresponding to thesearch mode identifier.
 8. The search system of claim 7, wherein thesynonym augmented search comprising searching using the first searchstring and at least one associated synonym.
 9. The search system ofclaim 8, wherein the differing meaning search comprising searching usinga plurality of meanings associated with the first search siring.
 10. Thesearch system of claim 7, wherein the differing meaning searchcomprising searching using additional text corresponding to at least oneof a plurality of contextual meanings associated with the first searchstring.
 11. The search system of claim 7, wherein the processinginfrastructure further receives restriction input from the first userand the offering further including a search limited by the restrictioninput.
 12. The search system of claim 7, wherein the search results arelimited by at least one adult filter parameter.
 13. The search system ofclaim 7, wherein the synonym augmented search is based upon a dictionaryapproach.
 14. The search system of claim 7, wherein the differingmeaning search is based upon a thesaurus approach.
 15. A search systemthat communicatively couples with a plurality of client devices thatdeliver a corresponding plurality of search strings, the search systemcomprising: a processing infrastructure that receives a first searchstring and a search mode identifier from a first device of the pluralityof client devices of a first user, the search mode identifier beingselected by the first user via an offering presented by the firstdevice, the offering including each of a first type of augmented searchand a second type of augmented search; the processing infrastructure,based on the search mode identifier, performs a search of text, the textbeing gathered in association with a web crawling process; and aninterface through which the processing infrastructure delivers searchresults to the first of the plurality of client devices, the searchresults corresponding to the search mode identifier.
 16. The searchsystem of claim 15, wherein the first type of augmented searchcomprising searching using the first search string and at least oneassociated synonym,
 17. The search system of claim 16, wherein the firsttype of augmented search comprises a dictionary based augmentation. 18.The search system of claim 16, wherein the second type of augmentedsearch comprising searching using a plurality of meanings associatedwith the first search string.
 19. The search system of claim 15, whereinthe second type of augmented search comprising searching usingadditional text corresponding to at least one of a plurality ofcontextual meanings associated with the first search string.
 20. Thesearch system of claim 19, wherein the second type of augmented searchcomprising a thesaurus based augmentation.
 21. The search system ofclaim 15, wherein at least a portion of the search results comprising aplurality of images.
 22. The search system of claim 15, wherein thesecond type of augmented search comprising a different meaning search.23. The search system of claim 15, wherein the processing infrastructurefurther receives restriction input from the first user and the offeringfurther including a search limited by the restriction input.
 24. Thesearch system of claim 15, wherein the search results are limited by atleast one adult filter parameter.
 25. A method performed by a searchsystem that communicatively couples with a plurality of client devicesthat deliver a corresponding plurality of search strings, the methodcomprising: receiving a first search string and a search mode identifierfrom a first device of the plurality of client devices of a first user,the search mode identifier being selected by the first user via anoffering presented by the first device, the offering including each of afirst type of augmented search and a second type of augmented search;searching, based on the search mode identifier, text gathered inassociation with a web crawling process; and delivering search resultsto the first of the plurality of client devices, the search resultscorresponding to the search mode identifier.
 26. The method of claim 25,wherein the first type of augmented search comprising searching usingthe first search string and at least one associated synonym.
 27. Themethod of claim 25, wherein the second type of augmented searchcomprising searching using additional text corresponding to at least oneof a plurality of contextual meanings associated with the first searchstring.
 28. The method of claim 25, wherein at least a portion of thesearch results comprising a plurality of images.
 29. The method of claim25, wherein the second type of augmented search comprising a differingmeaning search.
 30. The method of claim 25, further comprising limitingthe search results by at least one adult filter parameter.